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Abstract. We say that a word w on a totally ordered alphabet avoids the word 
V if there are no subsequences in w order-equivalent to v. In this paper we suggest 
a new approach to the enumeration of words on at most k letters avoiding a given 
pattern. By studying an automaton which for fixed k generates the words avoiding a 
given pattern we derive several previously known results for these kind of problems, 
as well as many new. In particular, we give a simple proof of the formula 20 for 
exact asymptotics for the number of words on k letters of length n that avoids the 
pattern 12 • • • (£ + 1). Moreover, we give the first combinatorial proof of the exact 
formula [HI for the number of words on k letters of length n avoiding a three letter 
permutation pattern. 

2000 Mathematics Subject Classification: 05A05, 05A15, 68Q45. 

1. Introduction 

In this paper we study pattern avoidance in words. The subject of pattern avoidance 
in permutations has thrived in the last decades, see [SO! and the references there. Only 
very recently Alon and Friedgut 3j studied pattern avoidance in words to achieve an 
upper bound on the number of permutations in Sn avoiding a given pattern. We study 
pattern avoidance in words by defining a finite automaton that generates the words 
avoiding a given pattern and use the transfer matrix method to count them. By this 
approach we are able to find the asymptotics, as n oo, for the number of words 
on k letters of length n avoiding a pattern as well as exact enumeration results. In 
particular we re-derive Regev's |21]j result on the exact asymptotics for the number 
of words on k letters of length n avoiding a pattern 12 •••(£ + 1), and give the first 
combinatorial proof of a formula for the number of words on k letters of length n 
avoiding the pattern 123. 

Let Sn denote the set of permutations of the set [n] := {1,2, ... ,n}. li a ^ Sk and 
r G S'n, we say that r contains a if there is a sequence 1 < ti < t2 <■■■< tk < n of 
integers such that for all 1 < z,j < A; we have T(tj) < T{tj) if and only if a{i) < cr(j). 
Here a is called a pattern. If r does not contain a we say that r avoids a. In the 
study of pattern avoidance the focus has been on enumerating and giving estimates to 
the number of elements in the set S'„(cr), the set of permutations in Sn that avoids a. 

1 



2 



FINITE AUTOMATA AND PATTERN AVOIDANCE 



Maybe the most interesting open problem in the field is: Does there exists a constant 
c such that [^^(r)! < c" for all n > 0? This problem is equivalent to the seemingly 
stronger statement, see jj^: 

Conjecture 1.1. (Stanley, Wilf) For any pattern r G 5*^, the limit 

lim |S'„(r)|", 

exists and is finite. 

The conjecture has been verified for layered patterns [Sj and for all patterns which can 
be written as an increasing subsequence followed by a decreasing jH]. In the latter 
reference Alon and Friedgut proved a weaker version of Conjecture namely: For 
any permutation a there exists a constant c = c{a) such that |5'„(cr)| < c"'^*^^\ where 
7* is an extremely slow growing function, related to the Ackermann hierarchy. The 
method of proof in [3] was by considering pattern avoidance in words. This is also the 
theme of this paper. 

Denote by [k]* the set of all finite words with letters in [k]. li w = W1W2 ■ ■ - Wg e [k]* 
and V = V1V2 ■ ■ ■ fr G [m]* where r < s, we say that w contains the pattern v if there is 
a sequence 1 < ti < t2 < ■ ■ ■ < tr < s such that for all 1 < i, j < s we have 

< if and only if Vi < vj. 

If w does not contain v we say that w avoids v. For example, the word w = 323122411 G 
[4]^ avoids the pattern 132 and contains the patterns 123, 212, 213, 231, 312, and 321. 
If S is any set of finite words we denote the set of words in 5" that avoids v by S{v). 

The history of pattern avoidance in words is not as rich as the one in permutations. 
We mention the references El El UHl UH 1201 • In [20] Regev gave a complete answer 
for the asymptotics for [[/cj'^d)^)! when n —>■ 00, where pe = 12 ■ ■ ■ [i + 1): 

Theorem 1.2 (Regev). For all k > i we have 

|[fc]"(p,)|^C,,fcn^(^-V (n^oo), 

where 

i=i j=i 

1.1. Organization of the paper. The paper is organized as follows. In Section |2] 
we present the relevant definitions and attain some preliminary results, and in Section 
El we use the transfer matrix method to determine the asymptotic growth for the 
sequence n 1— >• liA;]"'^))!. In Section Wl] we study the special features of the automaton, 
A{pe, k), which generates the words with letters in [k] that avoids the increasing pattern 
12 ■■■(£ + 1). Here we will give a simple proof of Theorem 11.21 using the transfer matrix 
method and give a combinatorial proof for the formula ^9] for |[/c]"(p)|, where p is any 
permutation pattern of length three. We also consider the diagonal sequence | [n]"'(123) | 
and determine its asymptotic growth as well as showing that its generating function 
is transcendental. We conclude the paper by indicating further problems connected to 
the work in this paper. 
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2. Definitions and preliminary results 

Given a word-pattern p and an integer A; > we define an equivalence relation ~p on 
[k] * by: f ~p w if for all words r G [k]* we have 

vr avoids p if and only if wr avoids p. 

For example, if p = 132, k > 4, v = 13 and w = 14, then v 'y^p w, since 133 avoids p 
but 143 contains p. At first sight it may seem difficult to determine if f ~p w, since a 
priori there is an infinite number of right factors r to check. By the following lemma 
we have to check only a finite number words r. 

Lemma 2.1. Let p be a pattern of length i and let v,w E [k]* be any two words. Then 
V w if and only if for all words r G [kY , < s < i, we have 

vr avoids p if and only if wr avoids p. 

Proof. Define an equivalence relation ~p on [k]* by: f ~p w if for all words r G [k]^, 

< s < we have 

vr avoids p if and only if wr avoids p. 

Clearly, v ^p w implies v ~p w. On the other hand if f w we may assume that 
there is an r G [k]* such that vr contains p and wr avoids p. Any occurrence of p in 
vr can use at most £ letters of r. Thus there is a subsequence r' of r of length at most 

1 such that vr' contains p and wr' avoids p, i.e., v '7<^'pW. □ 

Let S{p,k) be the set of equivalence classes of ~p. By Lemma l2.ll the number of 
equivalence classes is finite. We denote the equivalence class of a word w by (w). The 
equivalence classes of ~p for p E S3 and = 3, 4, 5 are given in Table 1. 

Definition 2.2. Given a positive integer k and a pattern p we define a finite automa- 
ton\ A{p,k) = {S{p,k), [k],6, {e),S{p,k) \ {{p)}), by 

• the states are, £{p,k), the equivalence-classes of r^p, 

• [k\ is the input alphabet, 

• 5 : S{p, k) X [k] — i> £{j), k) is the transition function defined by 6{{w),i) = {wi), 
where wi is w concatenated with the letter 2 G [k], 

• (e) is the initial state, where e is the empty word, 

• all states but (p) are final states. 

For an example see Fig. ^ 

We will identify A{p, k) with the (labelled) directed graph with vertices S{p, k) and with 

a (labelled) edge — ^ between {v) and {w) if vi ~p w. Clearly, we may order the states 
as Xi, X2, . . . , Xe so that ii i < j there is no path from Xj to Xi. The transition matrix, 
T{p,k), of A{p,k) is the matrix of size e x e with non- negative integer coefficients 
defined by: 

[T{p, k)]ij = \{s e[k]: S{xi, s) = Xj}\. 
Thus [T{p, k)]ij counts the number of edges between Xi and Xj, and T(p, k) is triangular. 
^For a definition of a finite automaton, see and references therein. 
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Table 1. The equivalence classes of ~p for p & S3 and /c = 3, 4, 5. 



Example 2.3. Ifp = 2314 and k = 5, then it is easy to check (see PIH1 ) that the states 
are (e) , (2), (3), (32), (34), (24), (23), (324), (341), (241), (234), (2342), (231), and 
(2314) (see Fig. Op. 




Figure 1. The figure shows the final states in the automaton ^(2314, 5). 
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Note that there are two edges between the states (324) and (241), namely (324) — > 

(241) and (324) — > (241). Moreover, all final states m .4(2314, 5) have 3 loops, except 
(324) which has 2 loops. 

The following simple lemma will be helpful in finding the asymptotic growth of the 
sequence for fixed k. 

Lemma 2.4. Let the automaton A{p, k) be given, let d be the number of distinct letters 
in p and suppose that k > d—1. If {v) is any state different from (p) , then the number 
of loops at (v) does not exceed d—1. Moreover, there are exactly d—1 loops at (e) . 

Proof. Suppose that there are more than d — 1 loops at (v). Then the loops use at 
least d different labels. From these labels we can form a word w order-isomorphic to 
p. But then vw ~p v which is a contradiction. 

Let pi be the first letter of p. Then, ii i < pi or i > k — d + pi we have i ~p e. But 
there are d — 1 such is, which proves the lemma. □ 

Although pattern avoidance in words and pattern avoidance in permutations share 
many common features, there are some important aspects in which they differ. For 
permutations there are three simple operations, /, that respects pattern- avoidance in 
the sense that /(r) avoids f{a) if and only if r avoids a, namely the reversal, the 
complement and the inverse of a permutation. The first two operations have obvious 
generalizations to words, while the inverse does not. It has in fact been an open question 
to construct an inverse for words possessing "the right" properties. Such an inverse was 
recently constructed by Hohlweg and Reutenauer |T^ . Unfortunately it is not possible 
to construct an inverse that respects pattern avoidance in words, which would imply the 
identity = for all k,n > and permutation patterns p. The first 

counter example to this is |[5]^(1342)| = 67854 > 67853 = | [5]^(1423)|, see Table lOl If 
w G [k]"' let the complement of w in [k]"' be = {k + l — wi){k + l — W2) ■ ■ ■ (k + l — Wn)- 
Then we have in fact that A{p, k) and A{p'^, k) are isomorphic as automata for any 
p E [k]*, since v w if and only if ~pc w'^. 

Certainly w avoids p if and only if w'' avoids p^, where is the reversal operator and 
w and p are any words. However A{p, k) and A{p^, k) are not in general isomorphic. 
Indeed, for p = 2314 and = 5 we have that |£(2314, 5)| = 13 and |f (4132, 5)| = 14. 



3. Transfer matrix method 



In this section we use the transfer matrix method (see [2S1 Theorem 4.7.2]) to obtain 
information about the sequences |[fc]"(p)|. Given a matrix A let {A;i,j) be the matrix 
with row i and column j deleted. If p is a pattern and > an positive integer let 
T'(j9, k) = {T{p, k),ek- 1, efc - 1). 

Theorem 3.1. Let k be a positive integer, p be a pattern and Ck be the number of states 
in A{p, k). Then the generating function for |[A;]"(p)| is 



n>0 



E;=i (-l)^'^^det(/-xr,j,l) _ deti?(x) 

n£T'(i-A.x) n£T'(i-A.x) 
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where Aj is the number of loops at state Xi, and B{x) is the matrix obtained by replacing 
the first column in I — xT' with a column of all ones. 

Proof. The theorem follows from the transfer matrix method, see Theorem 4.7.2], 
since we want to count the number of paths from {e) to any state other than {p) of 
length n in A{p, k). □ 

Regev [20] computed the exact asymptotics for |[A;]"(p£)|, where pi is the increasing 
pattern 12 ■■■(£+ 1) and n — >• oo. We will next find the exact asymptotics (up to a 
constant) for for all patterns p. Given two sequences {a„} and of real 

numbers, we denote a„ ~ 6„ if lim„^oo = 1. A path in A{p, k) is called simple if it 
starts at (e), does not use any loops, and does not end in {p). 

Theorem 3.2. Let p be any pattern with d distinct letters and let k > d — 1 be given. 
Then there is a constant C > such that 

~ 1)" (n^oo), 

where M + 1 is the maximum number of states with d — 1 loops, in a simple path. 

Proof. Let P simple path in A{p,k). Moreover, let ij be the 

number of loops at state Xj. Then |[A;]"(p)| = N{P,n) where 

N{P,n)= CC---^?, 

oiH \-aj=n—j+l 

and the sum is over all weak compositions of n — j + 1 into at most j parts. Now, 
N{P,n) is equal to the coefficient to in (1 - £it)~^ ■ ■ ■ (1 - £jt)~^ Let r be 

the number of i such that ii = d — 1. Note that by Lemma 12.41 r is greater than or 
equal to one. The dominant term of (1 — iit)^^ ■ ■ ■ (1 — ijt)~^ is (by partial fraction 
decomposition) equal to 

fit) 

where f{t) is a polynomial of degree less than r and f{{d — 1)^^) ^ 0. By well known 
results it follows that N{P,n) ~ C{P){d — l)"'n^'~^, where C{P) > is a constant 
depending on P and k. Taking the greatest possible r yields the desired results. □ 

When there are exactly d — 1 loops at every state except (p) in A{p, k), then it follows 
from Theorem 13.11 that |[fc]"(p)| = [d — l)"Q(n), where Q is a polynomial in n. We 
have in fact: 

Corollary 3.3. Let A{p, k) be such that all states but (p) has exactly d — 1 loops. Then 

M 

\mp)\ = j2<^,{d-i) 

j=0 

where Oj counts the number of simple paths of length j in A{p, k). Moreover, if p is a 
pattern of length i + 1 then aj = {k — d + ly for all j = 0,1, . . . ,i. 
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Proof. The corollary follows from the proof of Theorem 13.21 since N{P,n) = [d — 
If p is a pattern of length i + 1 then we have that aj = {k — d + 1)^ where 

j = 0, !,...,£, since ^ = J^j^Q ai{d - 1)^-' {^^ for all j = 0, □ 

As an example of Corollary 13.31 we note that if p is any pattern of length i + 1 with 
exactly d different letters then 

e 



\mp)\ = i2(^-ir~'(fj- 



4. The increasing patterns 



We will in this section investigate the properties of A{j)i, k), where p£ = 12 ■■■(£ + 1). 
The following lemma describes the structure of A{p£, k): 

Lemma 4.1. Let k > i be given. For any subset S of [k] of size £ let ws be the word 
consisting of the elements of S listed in increasing order. Then the words ws together 
with pe constitute a complete set of representatives for the equivalence-classes S{pe, k). 
In particular we have: 

\S{pe,k)\= 



If S = {si < ■ ■ ■ < Si} C [k] and j G [k] let 

= {si < ■ ■ ■ < <j< Si+i <■■ ■ < se}, 
where i is the integer such that Sj_i < j < Si, (sq := 0, s^+i := k + 1). Then 

{ws^i) if 3 < Se, 
{pij otherwise . 

In particular, the loops of ws are the elements of S. 



Proof. It is clear that the words ws are representatives for different classes. Let v G 
[k]*{p(). We say that an increasing subword xiX2---Xj of v is extendible if Xj < 
k + j — i — 1, i.e., if we may extend xiX2 ■ ■ - Xj to an occurrence of pe using letters from 
[k]. Suppose that the maximum length of an extendible increasing subsequence in v is 
equal to s, s < i. For 1 < j < s let 

rj{v) := mm{xj : extendible subword of f }. 

Clearly ri{v) < r2{v) < ■ ■ ■ < Tsiy). Let 

S = {ri{v), r2(f ), . . . , rs{v), k + s + 1 - i, k + s - i, . . . , k - 1, k}. 

Then we see that ws ~ v. The statement about the transition function follows from 
the construction. □ 



In the sequel we will use some standard notation from the theory of partitions and 
symmetric functions. For undefined terminology we refer the reader to Chapter 7 of 
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Theorem 4.2. Define a partial order on the final states in A{pe, k) by: x < y if there 
exists a path from x to y in A{p£, k). Then this partial order is isomorphic to 

J{[i]x[k-i]), 

the lattice of order ideals of the poset [i] x [k — i]. 

Proof. Let S = {si < S2 < ■ ■ ■ < Si} and T = {ti < t2 < ■ ■ ■ < ti} he subsets of 
[k]. We claim that there exists a path from (ws) to (wt) if and only if Sj > tj for all 
1 < i < £. From this the theorem follows since the latter poset is isomorphic to the 
interval [0, Xe,k-e], in the Young's lattice, where X^^k-e '■= {k — i, k — i, . . . , k — i) is of 
length i. Indeed, consider the bijection defined by: 

(si, 82,..., Si) H-^ (s£ - i, S£_i - £ + 1, . . . , Si - 1) G [0, Xe,k-£]- 

Then Si > ti for all 1 < z < j if and only if the image of S is greater than the image of 
T in [0, Xe^k-e]- But [0, Xe^k-e] is its own dual, so the statement follows from the simple 
fact that [0, Xi^k-i] is isomorphic to J([£] x [k — i]). 

If there is an edge between (ws) and (wt), we are done by Lemma f4. 11 The "only if" 
direction thus follows by induction on the length of the path. 

Now, if Si > ti for all 1 < i < i consider the path 

{ws) {Wsti) {Wstit2) ■■■ {Wstit2 ■■■ti). 

It is not hard to see that {wstit2 ■ ■ ■te) = {wt), which completes the proof. □ 

We now have a different proof of the following theorem of Regev ^ 
Theorem 4.3 (Regev). For all k > i we have 

\[kr{Pi)\^Ci,kn'^'-'k- {n^oo), 

where 

C^k=i'^'~'^tlU{^+J-l). 

i=l j=l 

Proof. By Corollarv 13.31 and Theorem 14.21 we have that 

\mpe)\ ^ «Mr"^(^)r ^ ^r^V^r {n 00), 

where M = i{k — i) and om is equal to the number of maximal chains in J{[i] x [A; — £]). 
By fI7\ Proposition 7.10.3] and the hook-length formula j27| Corollary 7.21.6] we have 
that 

ank-i) = / " ^ = ^^-^ , 

i=i j=i 

from which the theorem follows. □ 

It should be clear from the correspondence in Theorem 14.21 that the simple paths of 
length r in A{pe, k + i) are in a one-to-one correspondence with tableaux T of the 
following type: 
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(i) T is weakly increasing in rows and columns, 

(ii) no integer appears in more than one row, 

(iii) the entries of T are exactly [r], 

(iv) the shape of T is confided in Xi^k- 

Recall that the tableaux satisfying (i) and (ii) above are the border-strip tableaux (or 
rim-hook tableaux) of height zero. We call these tableaux segmented. Let a{i,k,r) 
denote the number of segmented tableaux satisfying (iii) and (iv), so that: 

(4.1) |[fc + £]"(p,)| = J]r-'-a(^,A;,r) 

r=0 

The function a{i, k, r) is actually a polynomial in k of degree r. To see this let us call 
a segmented tableau inside [£] x [k] primitive if all columns are different, and let the 
set of such tableaux of length i with r different entries be VTZi^i^r- If we denote the 
number of elements in VTZe^i^r by pr(i', i, r) we have 

r 

a{i, k, r) = pr(^, i, r) 

i=r/l 

since for any such primitive tableaux of length i we may insert a number ai copies 
of the first column before the first column, a number 02 copies of the second column 
between the first and the second column, and so on. After the last column we may 
insert a number ttj+i columns of all blanks, requiring that 

«! + ^2 + ■ ■ ■ + C^i+l = k — i. 

Thus there are (*^) segmented tableaux arising from a given primitive one. The numbers 
pr(£, i,r) are in general hard to count, but there are two special cases which are nice, 
namely pr(£, r, r) and pr(2,'i,r). We start by counting pr(£, r, r). 

Theorem 4.4. With definitions as above: 

pr(£,n,n) = \Sn{pe)\. 

Proof. We will define a bijection between Sn and U£>o'P7^£,n,n such that the height of the 
tableau corresponds to the greatest increasing subsequence in the permutation. Recall 
the definition of rj(f) in the proof of Lemma 14. 11 and let r{v) = {ri{v), r2{v), . . . , r^(f)), 
where i is the length of the longest increasing subsequence in v. Let k be big enough 
so that all increasing subsequences in permutations in Sn are considered extendible. 

Now, if TT = 7ri7r2 ■ ■ ■ 7r„ is any permutation in Sn define T = T{tt) as follows. Let the 
first column of T be r(7r) the second column be r(7ri ■ ■ ■ 7r„_i) and so on. The image of 
the permutation 351462 is: 

11113 3 
T(351462) =24455 
6 6 

By Lemma l4.ll we have that T(7r) G VTZe^n,n- Moreover from Lemma l4.ll we also get 
that a tableau T is the image of some vr G S'„ if and only if 

(a) T has n columns and entries 1, 2, . . . , n. 
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(b) Let T* denote the ith column. If i < j then is smaller than in the product 
order. (If T* and have different size fill the empty slots of with n + 1), 

(c) Exactly one new entry appears every time you move from T*^^ to T*. 

Now, if T G Ue>oV7le^n,n condition (a) and (b) are trivially satisfied. At least one new 
entry appears every time we move from T*"*"^ to T\ since otherwise T* = T*"*"^ and T 
fails to be primitive. On the other hand if there appear more than one new entry in 
a transition then in a later transition there must appear no new entry, since T has n 
columns and n entries. This verifies condition (c) and the theorem follows. □ 

A special case of Theorem l4.4l is that pr(2, n, n) = Cn, the nth Catalan number. This is 
also a special case of the next theorem. Note, that Theorem 14. 51 is what we need to have 
combinatorial proof of a closed formula, see Theorem 14.71 for the numbers |[/c]"(123)|. 
Burstein jO] achieved a different, but of course equivalent, formula for |[fc]"(123)|, but 
not in a bijective manner. 

Theorem 4.5. With definitions as above: 

pr(2,^,r) = -^f^.'V ' Y 

Before we give a proof of Theorem 14.51 we will need some definitions and a lemma. 
Let V7l'^{2, s, r) be the tableaux in V7l{2, s, r) that fill up the shape [2] x [r], and let 
pr+(2, s, r) := \V7l^{2, s,r)\. Then pr(2, s, r) = pr+(2, s, r) + pr+(2, s,r + 1) since we 
get the tableaux that do not fill up the shape by deleting all entries r + 1. To prove 
the theorem we will show that pr"'"(2,s,r) = (g'^^^JC's, where Cs is the sth Catalan 
number. 

We first define an operation + that takes tableaux with r different entries to tableaux 
with r + 1 different entries. Let T G V7l'^{2, s, r). Suppose that j is an index such that 
Tij = for some i = 1,2. Write T as T = LR where L is the j first columns and 

R is the s — j last columns. Let R' be the array order equivalent with R with entries 
the same as R, add r + 1, take away Ti(^j+i) (two arrays A and B are said to be order 
equivalent if Aij < Ai/j/ if and only if Bij < Bi'j' for all i,j,i',j'). We define T + j to 
be the tableaux T + j := LR'. In T there are exactly t = 2s — r indices j G [s — 1] such 
that Tij = Tj(j+i) for some i = 1,2. Let S = {si < S2 < ■ ■ ■ < St} he these indices and 

define a function $ : V7l'^{2, s, r) (''^7^') ^ '^^2,s, where ST2,s is the set of standard 
tableaux of shape [2] x [s], by 

^T) = {S,T + St + St-, + --- + s,). 

The fact that $ is a bijection will prove the theorem, since by the hook-length formula 
we have |iST2,s| = Cs. To find the inverse of $ we need a kind of inverse operation to 
+. 

Let T G V7l'^{2, s,r) and l<6<s — Ibe such that Tib < and T2b < T2(b+i)- 

Define two arrays T\b and T\^ a.s follows. Write T = LR where L are the b first columns 
and R are the s — b last columns. Define := L'R', to be the array where L = L' 
and R' is the unique array order equivalent with R, with entries the same as R add Tib 
take away r. Similarly, let T\b := L'R', be the array with L = L' and where R' is the 
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unique array order equivalent with R, with entries the same as i?, add T25 take away 
r. 



1 2 4 4 ^ 
3 5 6 7 



12 4 4 
3 5 6 7 ^ 

|2 



12 2 2 

3 5 4 6 

12 4 4 

3 5 5 6 



Note that exactly one of T| and TI2 above is a primitive segmented tableaux. This is 
no accident. 

Lemma 4.6. Let T G VTZ'^(2, s,r) and 1 < b < s — 1 be such that Tih < Ti(f,_|_i) and 
T2b < T2(b+i) ■ Then 

T|b G P7^+(2,s,r- 1) ^ ^ P7^+(2,s,r- 1) 

^2(fe+l) = ^26 + 1 

Moreover, zf B = T\^ G P7^+(2,s,r - 1) then Bib = ^1(6+1) and if A = T|b G 
P7^+(2, s, r - 1) t/ien = A^b+i). 

Proof. Consider A := T|f,. All entries in T that are smaller than will be mapped 
on themselves and Aij = — 1 for Aij > T2b- Therefore A G VTZ~^{2, s,r — 1) if and 
only if T2(f,+i) = T2b + 1 (since otherwise the entry T2b will appear in both the first and 
the second row). 

Consider B := T\^. Let i/i, i = 1, 2, . . . , /i be the entries in T satisfying T26 < Ui < 
^2(6+1) ordered by size. Then the entry yi will be mapped to an element smaller than 
T2b and Ui will be mapped to for i > 1. Thus B G V1Z'^{2, s, r — 1) if and only if 
^2(fe+i) > ^26 + 1 as claimed. 

The last statement is a direct consequence of the above proof. □ 
We are now ready to give a proof of Theorem 14.51 

Proof of Theorem \4.5\ If T G V1Z'^{2, s, r) and 1 < 6 < s — 1 are such that T15 < Tup+i) 
and T2b < ^2(fe+i) we define T — 6 to be the one of the arrays T\b and T\^ which is in 
VTZ^{2, s,r — 1). By Lemma [4.61 we have that 

(4.2) (T + j) -j = T if Tij = Ti^j+i) for some 2 = 1,2, 

{T - j) + j = T if Tij < Ti(j+i) for both i = 1,2. 

Now, if = {a;i < X2 < ■ ■ ■ < Xt}, where t = 2s — r and P G ST2,s we let 

^{S,P) ■=P-xi-X2 Xt. 

By ()4.2|) it follows that ^ is the inverse to $ and the theorem follows. □ 



We now have a combinatorial proof of the following theorem given in a different form 
in [nj: 
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Theorem 4.7. For all n, k > we have 

/ k 



|[A; + 2]"(123)| = ^2"-'-C, 



r — % \r 



•where Ci is the ith Catalan number. The generating function 

F{x,y) :=5^|[A; + 2r(123)|xV, 



xy{l-y) 



n,k 

is given by 

^^^'^^ = {l-x){l-2yf V(l - - '2y)\ 
where C{z) is the generating function for the Catalan numbers. Equivalently, F{x,y) 
is algebraic of degree two and satisfies the equation: 

x{l - x)y{l - y)F^ - (1 - x)(l - 2y)F +1 = 0. 



To complete the picture for permutation patterns of length 3 it remains to enumerate 
|[A;]"(132)|. Simion and Schmidt |211 introduced a simple bijection between S'„(123) 
and S'„(132) which fixes each element of S'„(123) U S'„(132). West [221 generalized this 
bijection to obtain a bijection between Sn{p) and 5'„(g) where p{i) = q{£ — 1) = i, 
p{i — 1) = q{i) = i — I, and p,q E Si. Here we indicate how to generalize West's result 
to obtain a bijection between and where p and q are as above. 

Theorem 4.8. Let p = piP2---pe. be a pattern with greatest entry equal to d and 
Pi-i = d — 1, Pi = d. If d occurs exactly once in p then 

\mp)\ = \mp}i 

where p = pip2 ■ ■ ■ PiPe-i ■ 



Proof. The proof is a straight forward generalization of West's algorithm presented in 
Section 3.2]. 

□ 



For example, if p = 132 then p = 123. Hence, by Theorem 14.81 we get that if p and 
q are any permutation patterns of length 3 then |[/c]"(p)| = for all n,k>0 

(see [9j for an analytical proof). If p = 1232 the p = 1223. Hence, Theorem 14.81 gives 
|[fc]"(1232)| = |[fc]"(1223)| for aU n,k>0. 

Since, Sn{p) C the numbers are interesting. A sequence f{n) is 

polynomially recursive (P-recursive) if there is a finite number of polynomials Pi{n) 
such that 

AT 

Y^P^{n)f{n + t) = 0, 

for all integers n > 0. For the case when p is permutation pattern of length 3 we have 
the following: 
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Theorem 4.9. Let p be a permutation pattern of length 3. Then the sequence f{n) : = 
is P -recursive and satisfies the three term recurrence: 

p{n)f{n — 2) + q{n)f{n — 1) + r{n)f{n) = 0, 

where 

p(n) = 3(n-3)(n- l)(3n-5)(3n-4)(5n-4), 

g(n) = 288 - 1440n + 2780n2 - 2435n^ + 976n^ - 145n^ and 

r{n) = 2(n-2)2n(n + l)(5n-9). 

Proof. The fact that f{n) is P-recursive foUows easily from the expansion of f{n) as a 
double sum using Theorem 14.71 and the theory developed in ^\ . The polynomials p, q 
and r were found using the package MULTISUM (see (2H1) developed by Wegschaider 
and Riese. □ 

Corollary 4.10. The asymptotics of f{n) = |[n]"(123)| is given by 
where C > is a constant. 

Proof. This is a direct consequence of Theorem 14.91 and the theory of asymptotics for 
P-recursive sequences, see |3J. □ 

A consequence of this is that the generating function of f{n) is transcendent, since the 
exponent of n in the asymptotic expansion of a sequence with an algebraic generating 
function is never a negative integer. 



4.1. Generating function approach. In this section we will investigate the gener- 
ating function that enumerates the number of segmented tableaux according to size 
of rows and number of different entries. Let A£{xi,X2, ■ ■ ■ ,xe,t) be the generating 
function: 

A _ \^ Ai(T) Ai(r)-A2(T) \e-i{T)-\e{T).N{T) 
T 

where Aj(T) denotes the size of row i in T, N{T) denotes the number of different 
entries in T and the sum is over all segmented tableaux with at most i rows. For 
i = 1, 2, . . . , £ let v4^(xi, . . . ,X£,t) be the generating function for those tableaux which 
have their maximal entry in row i. If F{xi,X2, ■ ■ ■ ,Xn) is a formal power-series in n 
variables the divided difference of F with respect to the variable Xj is 

F- Fixi = 0) 
AiF := ^, 

Xi 

where F{xi = 0) is short for F{xi, X2, ■ ■ ■ , Xi^i, 0, x^+i, . . . , Xn)- 
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Theorem 4.11. With definitions as above we have that Ai satisfies the following system 
of equations: 

Ae = 1 + AI + --- + AI 

A] = xiX2tAi + X1X2AI, 



Aj = XgtAsA^ + 



£5 



A^-^ = XitA^^iAi + xeAe_iAl-\ 
A\ = tAeAi + AiA\. 

Proof. The theorem foUows by treating two separate cases. Let n be the greatest entry 
in the tableau T. The case when there is one n in a row corresponds to the first 
summand and the case when there are more than one n in a row corresponds to the 
second summand. □ 

When i = 2, A = A2, the system boils down to: 

(4.3) ((1- Xo^)(l - ^1^) _ XoH] A = 1- xrVl + t)A(x2 = 0). 
V 1 - X1X2 J 

This equation can be solved using the so called kernel method as described in If 
we let 

_ 1 + + 2t) - v/(l + + 2t)y - 4xi{l + t)^ 

2x,{l + t) 

then the parenthesis infront of A in ()4.3p cancels, and we get: 



1 + xi(l + 2t) - V(l + + 2t))2 - 4xi(l + ty 

A{x2 = = . 

2xi(l + 1)^ 

By the interpretation of a{i, k,r), we have that the bi-variate generating function for 
a(2, k, r) is {l + xi)~^A2{xi, 1, t). From this and ()4.1|1 one may derive an analytic proof 
of Theorem 14.71 

5. Further results and open problems 

5.1. Further directions. Recall that the Stanley- Wilf Conjecture asserts that for 
any permutation vr the limit lim^^oo \Sn{'^)\^^'^ exists and is finite. What about the 
sequence |[n]"(7r)|? 

Problem 5.1. Let n be a permutation. Is there a constant < C < 00 such |[?2]"(7r)| < 
for alln>0? 

Note that the answer to Problem 15.11 is no when vr is not a permutation, since then 
»5'n(7r) C [n]"(7r). Again, Problem 15. II is equivalent to the statement that 

lim |[n]"(7r)|i/", 

exists and is finite. This is because for all m, n > we have 

|[n + m]"+"^(7r)| > |H"(7r)| ■ |[m]™(7r)|. 
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SO we may apply Fekete's Lemma on sub-additive sequences. See jl] Theorem 1] for 
details (the proof extends to words word for word). For permutations tt E S3 we have 
by CoroUarv lCTH that lim^^oo IM"(vr)|i/" = 13/2 as opposed to lim„^oo |5„(7r)|^/" = 4. 

For which permutations do we know that Problem 15.11 is true? It follows from the 
work in !^ that Problem 15.11 is true for all permutations which can be written as an 
increasing sequence followed by a decreasing. Also, with no great effort Bona's proof 
|H1 of the Stanley- Wilf conjecture for layered patterns may be extended to this setting. 
Thus for all classes that the Stanley- Wilf conjecture is known to hold, the seemingly 
stronger Problem 15. II holds. The following conjecture therefore seems plausible: 

Conjecture 5.2. For all permutations it we have: 

3CVn(|[n]"(p)| < C") ^ 3Z}Vn(|^„(p)| < 

There are several problems concerning the automatons associated to a pattern that 
has connections to the above problems. One problem is to give an estimate to the 
number of simple paths in A{p, k), another is to estimate the number of equivalence 
classes in A{p, k). Yet another problem is to give an estimate to the maximum size of 
an equivalence class. 

5.2. Formula for |[A;]"'(p)|. Our algorithm (see Theorem 13. 1|) for finding a formula for 
|[A;]"'(p)| is implemented in C++ and Maple, see |18j. The first with input p and k 
and output the automaton A{p, k) and the second with input the automaton A{p, k) 
and output the exact formula for |[/c]"(j9)|. This algorithm allows us to get an explicit 
formula for |[fc]"(p)| where p E and k >1 are given. For example, an output for the 
algorithm for p E and k = 3, 4, 5, 6 is given by Table 1^121 where we define. 




Finally we remark that our method can be generalized as follows. Given a set of 
patterns T we define an equivalence relation ~t on [k]* by: v ~t w if for all words 
r G [k]* we have 

vr avoids T if and only if wr avoids T, 

where a word u avoids T if u avoids all patterns in T. As in Section 2 we define an 
automaton A{T, k) with the equivalence classes of ~r as states. With minor changes in 
the proof. Theorem 13. II can be extended to avoidance of a set of patterns. For example, 
if T = {1234, 2134} and = 5, then by [H] we get that 

|[4]"(r)| = 2-3" + 2Q)3"-2 _ 2^^, 

|[5]'^(r)| = 3-3" + 6(2)3"-2 + 6(3)3"-3 + 8Q)3"-4 _ 2 . 2", 

|[6]'^(T)| = 4 ■ 3^^ + 12(^)3"-2 + 24g)3"-3 

+54(^)3"-^ + 60 g) 3"-^ + 40(g)3"-6 - 3 ■ 2". 

Acknowledgements. We would like to thank M. Bousquet-Melou and C. Kratten- 
thaler for interesting discussions, D. Zeilberger for informing us about the work of 
Wegschaider and Riese, and Wegschaider and Riese for running their program for us. 



16 



FINITE AUTOMATA AND PATTERN AVOIDANCE 



p 


K 


m KP) 


1234, 1243 


3 


[1]3 


1432, 2143 


4 


[1,1,1,1]3 




5 


[1,2,4,8,11,10,5]3 




6 


[1, 3, 9, 27, 66, 126, 183, 189, 126, 42]3 




7 


[1, 4, 16, 64, 221, 632, 1478, 2772, 4074, 4536, 3612, 1848, 462]3 




8 


[1, 5, 25, 125, 555, 2103, 6735, 18075, 40290, 73770, 109206, 






iz M iU, iiooDU, /z9o(J, oUUoU, bUUoJs 




o 
y 


[1 Qfi olfi 1 1 7n c;c;nfi ooqqs wfiss OQnsoQ c;fiQ/iin 
[i, 0, oD, ZiO, ii ( U, OOUo, zzooo, ( ( Ooo, zoUozo, 0oo4iU, 






10/I7n7fi OO'^'^SIR '?'?008'?R /inO'^7'?R '^SSO'^n'^ 98fi7/l'?R 
iZ4 / U / D, ZZOOOiD, OOZZOOD, 4UZ0 I OD, OOOUOUO, ZoD / 40D, 






1 F^OQnc;^^ c;oc;nn^^ q'tc^i^^I 
lozoyoo, ozoUyo, o /oibja 




lU 


[1, / , 4y, o4o, Ziyi, izoio, oUool, zo /4U / , yooD04, oUo4ooo, OOZ/bDD, 






on/ioo/i^^o /loo^^otxo/i 7/1/1 oo 7o iinni^^nni 10 vnnoo^^ 1 
zU4oz4oz, 4zzoooo4, ^44ozo^o, iiUyioUyi, io/yyoobi. 






140882742 115068954 72390318 32978946 9699690 1 3856701 q 


1 QO/I 
ioz4 


Q 

o 















h 9/1811 infill 
[1, 2, 4, 8, 11, lU, £), IJ3 






1 3 9 27 66 126 183 197 152 80 26 41, 




7 


1 4 16 64 221 632 1478 2808 4308 5295 5152 3895 2219 






904 239 33 lli 




Q 
O 


ni 

[^J3 




4 


f1 1 1 11q 

[±, ±, X, XJ3 




5 


1 2 4 8 11 10 4U 




6 


1 3 9 27 66 126 176 168 96 24U 




o 
o 


\^ 1 
[-LJ3 




Zl 


ri 1 1 iio 




5 


[1]2 + [0.3.3,9,10,11,3]3 




6 


[13, 1]2 + [-12, 15, -2, 37, 57, 134, 169, 167, 76, 12)3 


2413 


3 


[1]3 




4 


[1,1,1,1]3 




5 


[10,4, 1]2 + [-9, 8, 1,9, 11,10,2)3 




6 


[96, 28, 5]2 + [-95, 71, -36, 54, 52, 132, 167, 137, 44, 4)3 



Table 2. Patterns of length 4 
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